A Syntax-based Rule-base for Textual Entailment and a Semantic Truth Value Annotator

Textual entailment (TE) is the semantic inference task which takes two text fragments, and determines whether one entails the other. It captures the semantic inferences needed by many text understanding applications. Practical TE applications usually adopt relatively low-level lexical or lexicalsyntactic representations of text, which correspond closely to language structure. In many cases, such approaches miss out on some of the valuable and more abstract information, at the generic-syntactic and semantic levels. This thesis first presents a novel comprehensive generic syntax-based rule-base for the TE task, which offers a wide variety of over 60 generic entailment rules that can substitute between copious equivalent or entailing constructions in categories such as: active vs. passive, coordination, apposition, determiners, possessives, and case correction. Also, they can extract simplified IS-A and HAS-A implications from over a dozen generic patterns, and successfully decouples relative clauses. Based on inputs from previous salient works on the topic, as well as novel contributions, the rule-base is represented according to the Stanford Dependencies standard, is based on a well defined formalism, and is made publicly available for use with TE systems, including full documentation, and tools for rule design and software compilation. The rules are honed for high resilience to syntactic structure diversity, especially in RTE texts, which are used in the field’s most common benchmark. Qualitative and quantitative depth evaluations are reported, including a manual dataset analysis, that demonstrate the high potential for knowledge resources of this type in TE, the wide coverage this rule-base has over the required set of syntax-based transformations in this setting, its high accuracy, and how it significantly improves the performance of a concrete entailment engine. Additionally, I show that much work is needed for such systems to make better use of syntax-based resources. I then introduce TruthTeller, a novel algorithm and system that takes the syntactic parse tree of a given sentence, and identifies the semantic truth value of each predicate and clause. In contrast with previous work, it is the first such system meant to serve the natural language inference research community as a an open source tool, enriching conventional syntax tree representation. Some possible uses for these annotations are: inferring parts of a sentence from the whole, improving similarity (and contradiction) measures between texts, and improving the accuracy of entailment rule matching. As a side product, TruthTeller also annotates negation, and a classification of predicate implication types (factive, implicative, etc.). The download package also includes a lexicon of predicates and their implication types, the largest of its kind and the first to be made publicly available. Both the lexicon and TruthTeller’s annotations are shown to have good accuracy, recall and precision.